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Abstract 

In Requirements Engineering, requirements elicitation aims the 
acquisition of information from the stakeholders of the system-to-be. 
An important task during elicitation is to identify and render explicit 
the stakeholders' implicit assumptions about the system-to-be and its 
environment. Purpose of doing so is to identify omissions in, and con- 
flicts between requirements. This paper offers a conceptual framework 
for the categorization of implicit assumptions that stakeholders may be 
making during elicitation. The framework is relevant for practice, as it 
is a checklist for types of questions to use during elicitation. The con- 
ceptual framework is based on empirical research into default rules 
that individuals apply during decision-making. 
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1 Introduction 



Requirements elicitation is the acquisition of information from the stakeholders of the 
system-to-be. This information is a source of requirements that the system-to-be should 
satisfy. Requirements elicitation is one of the first steps of Requirements Engineering 
(RE), the main purpose of which is to produce a specification of the system-to-be, 
which satisfies the requirements, and which is sufficiently clear, precise, and complete 
to be used in subsequent systems engineering steps. Hereafter, we will refer to Require- 
ments elicitation only by elicitation. 

Elicitation involves communication with the stakeholders. Although communica- 
tion is not the only means to elicit information relevant for RE, we will focus only on 
it in this paper. 

Information that stakeholders provide can be uncertain and incomplete. Uncertain, 
because it reflects their beliefs and desires about the future. Incomplete, because they 
cannot anticipate all conditions that may arise in the future, when the system-to-be is 
operational. Our concern in this paper is completeness, while we will not be discussing 
uncertainty. Regardless of how uncertain information from the stakeholders is, our aim 
is to look for how to reduce its incompleteness. 

In particular, our concern in this paper is how to acquire information that is implic- 
itly assumed by stakeholders during elicitation. We will provide a conceptual frame- 
work, through which we identify types of implicit assumptions that ought to be ren- 
dered explicit during elicitation. 

Our starting point is the idea that implicit assumptions that stakeholders make 
can be understood as default rules in non-monotonic reasoning. Influential theoretical 
frameworks have been proposed to model non-monotonic reasoning [7 8 9 §)• 

In this paper, we use Reiter's default logic as a conceptualization through which 
to study implicit assumptions stakeholders may be making during elicitation. We con- 
sider that such assumptions are defaults, the normality assumptions that stakeholders 
consider given, and from which they derive information which they then explicitly 
communicate. We call the set of such assumptions the elicitation context. 

We propose a taxonomy for the elicitation context and establish a clear distinc- 
tion between requirements that are communicated, and those that are only assumed 
by stakeholders. Based on preliminary studies, we propose a discussion on how the 
non-monotonic reasoning-based formalization of decision making happening during 
elicitation is influenced by the definition of the elicitation context. We provide a con- 
ceptual framework which acts as a definition of the elicitation context. In practice, such 
a definition can be used as checklist for questions to use during elicitation. 

2 Organizational and Elicitation Context 

The application of default logic to elicitation requires to distinguish between different 
contexts that exist during elicitation. By definition, the term stakeholder refers to more 
than one actor, and consequently to more than one context. Defining the borders be- 
tween these contexts is essential to engineers, since what is true in one context may not 
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be verified in a second one. Figure [T] illustrates the different contexts that exist during 
the elicitation with two stakeholders a and j3. 

Communication Context 

= (U"=i Context^ U Context Eng U Context^ 



System Context 




Elicitation Context 



Figure 1: The Communication Context - Stakeholder's versus Engineers' Context 



The elicitation context refers to any information regarding the future system that 
is not accessible to stakeholders, i.e., on which stakeholders have to make default as- 
sumptions. Elicitation context combines both system's and engineers' contexts. Given 
such representation, elicitation is a set of informational exchanges between engineers - 
who are "owner" of the information system's context - and stakeholders' contexts. By 
"owner", it is meant that the actor is not part of the context, but perfectly knows it and is 
the only one to do so. Figure |2]represents the communication context taxonomy using 
UML standards. The diagram should be read in terms of informational contents, e.g., 
the communication context is composed of information about the elicitation context 
and contexts of stakeholders. 

Based on Figure [T] we see two stages in elicitation: (i) identifying ground require- 
ments and (ii) rendering explicit default requirements. The first stage is built upon 
regular elicitation methods - interviews, surveys, case studies, etc. - to collect regular 
requirements, that we name ground requirements. The second stage should find a way 
to render explicit assumptions that are made by stakeholders, but which are not explic- 
itly communicated. We name such assumptions default requirements . For an elicitation 
to be complete, an engineer should account for both ground and default requirements. 

For instance, the first stage could consist in asking a stakeholder to explain what 
features she wants a future system to have. She will certainly explicitly give a ground 
requirement under the form "I want the system to share data". The requirement to share 
data is given on the basis of a default theory. Doing so, she introduced various implicit 
assumptions that she considered true by default and that she did not communicate ex- 
plicitly, even though it may be important to the engineers. The second stage should 
consequently focus on the identification of these assumptions - or default require- 
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Figure 2: The Communication Context - Class Diagram Representation 



ments. Since defaults are built upon stakeholder's own context, default requirements 
are directly related to context's definition. To account for such context influence, we 
propose that stage two of elicitation should produce questions using dimensions of our 
context framework. 



3 Default Requirements 

The large range of non-monotonic theories offers many different ways to formalize 
and study reasoning and decision making in presence of defaults. We consider Reiter's 
default logic 1 9 1 in our paper. We argue default logic is adequate to study RE decision 
making because it proposes a set of concepts that corresponds to important aspects of 
elicitation. The purpose of Reiter's default logic is to formalize inference rules with- 
out explicitly mentioning all their exceptions, i.e., formalize inferences by default of 
contradictory information. In default logic, the normality assumption of a stakeholder 
states that, in absence of evidence to the contrary, default assumptions hold. This is 
typically what a stakeholder does whenever she has to make the decision of communi- 
cating a requirement facing an elicitation context she does not know: she uses a default 
theory. 

A default theory is a pair (D, W), in which D is a set of default rules {Di, D n }, 
and W is background knowledge. In our case, we focus on the default theory of a stake- 
holder a: W a is consequently a set of first order logic (FOL) premises that summarizes 
what the stakeholder knows "for sure" about communication context. Since Context a 
is the only context accessible to the stakeholder, W a should only contain elements that 
follows from Context a . As an illustration, consider a stakeholder who is used to an 
old inventory management software called S'icfclOO. The stakeholder has some knowl- 
edge about this part of the context: it can connect to the Internet, it is open source, it 
can exchange data, etc. This background knowledge is only known by the stakeholder, 
is limited to the borders of the stakeholder's context, and is represented formally as: 
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W a : [Context st 



akeholder} 



(i) 



supportsTCP(StcklOO) , 
-^proprietary(StcklOO), 
shareData(StcklOO) , 
-ibackupSys(StcklOO), ... 

The background knowledge of engineers on the communication context can be 
richer. For instance, they may have some knowledge about the context of other stake- 
holders based on former informational exchanges. Note here that this knowledge may 
be different from the knowledge W a and Wp that stakeholders may respectively have 
about their own context. The knowledge of engineers may also comprise informa- 
tion that are not issued from stakeholders' context but are part of the communication 
context. For instance, engineers may already have knowledge about the system, e.g., 
they may have an idea about the future software they might want to use - namely 
Account200, they may know it supports TCP/IP and is a proprietary software. They 
may also have knowledge that is not related to the information system but rather to their 
own context. As discussed in previous section, the combination of information system 
and engineer context forms the elicitation context. The same applies for knowledge 
background of engineers on elicitation context. 



WElicitation Context — { {W/S Context} > {WEngineer Context} } 



(2) 



The general background knowledge of engineers is broader than W a , is limited to 
the borders of the communication's context, and is represented formally as: 



{{Wcontext a} , 
{Wcontext fi} , 
{W Elicitation Context} 

We name D a the set of default assumptions that the stakeholder a makes. A default 
assumption is used by the stakeholder to decide what information to share, despite 
her limited understanding of the elicitation context. Consider for example the design 
of a new information system. The first iteration in the elicitation process consists in 
collecting ground requirements of the stakeholder, e.g., after a first interview, engineers 
learn that the stakeholder wants the future system to have the feature of sharing data 
with other softwares. What they are not supposed to be aware of is that the stakeholder 
knows - based on W a - that it is possible for a system to share data if it supports 
TCP/IP and is open source because StcklOO has such characteristics and can share 
data. Consequently, by default of elements that go against this assumption, any other 
system that is known to have such characteristics is assumed by the stakeholder to be 
able to share data. The default Di of the stakeholder can be represented formally as: 



^ ( support sTCP(X) A ^proprietary '(X) : M shareData(X) 1 
1 \ shareData(X) J 

Di can be read as follows: for any X that supports TCP/IP and which is not pro- 
prietary, and if it is relevant that X can share data, then the stakeholder believes X 
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can share data. SupportsTC P() and -^proprietary^ are prerequisites to the de- 
fault: they are necessary but not sufficient for the default to hold. M is a consistency 
test: if it is verified, the consequent shareData(X) is believed by the stakeholder. 
: M shareData(X) is said to be consistent with W a if -^shareData(X) does not 
follow from W based on deductive inferences. 

Considering the two different knowledge backgrounds W a and W Engineers, the 
problem of elicitation can be reduced to a problem of consistency test. Considering 
X = Account200, default Di is consistent with W a , i.e., ^sharedata(Account200) 
does not follow from W a . However, the stakeholder's default Di does not hold based 
on WEngineers because engineers know proprietary(X = Account200). Actually, 
there is a conflict between the ^proprietary(X = Account200)'s prerequisite of the 
stakeholder and the knowledge WEngineers which states proprietary(Account200). 
Such conflict exists because the information system's context is owned by engineers, 
and is oblivious to stakeholders. If engineers do not pay enough attention to identify 
those prerequisites, this conflict may not be identified and the requirements documen- 
tation may be incomplete. In order to render explicit default requirements, engineers 
should identify and discuss with stakeholder each aspect of the elicitation context. 

In a more informal way, a ground requirement such as "I want the new software X 
to have the feature of sharing data" presupposes for a stakeholder to consider prerequi- 
sites such as "the software X supports TCP/IP and is open source". Previous example 
is a typical illustration of some assumptions made by the stakeholder, that are not nec- 
essarily explicitly communicated to engineers, but which actually constitute important 
requirements: without such prerequisites, the default of the stakeholder does not hold 
and the requirement is not respected. Default requirements are parts of the needs of the 
stakeholder, but are not always communicated because considered by the stakeholder as 
holding true in the elicitation context, by default of contradictory information. If those 
default requirements are not verified, the conclusion built using the default assumption 
is destructed. Hence, the ground requirement shareData(X) should be retracted if the 
elicitation is not complete enough, e.g., if it appears that the new software is going to 
be proprietary and closed to connection with other software. 

During elicitation, engineers should know what default requirements underline a 
communicated ground requirement, since knowing the default requirements helps to 
decide if the ground requirement that the stakeholder communicated is correctly de- 
fined. Doing so without understanding the structure of the decision process is difficult. 
With default logic, we can establish a clearer distinction between what is presupposed 
by the stakeholder and what is actually communicated. We can read a default in re- 
quirements elicitation as follows: 

( default requirements : consistency test 
\ ground requirements 

Based on equation [5] engineers know that the ground requirement which is com- 
municated is not complete enough to be reported "as is" in the requirements documen- 
tation. To make sure the elicitation is complete, engineers must identify prerequisites 
of the default, that is default requirements. We argue in the next part of this paper that 
such identification of default requirements can be performed using the support of a 
context framework. 
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4 Literature Review 



The study of context and its influence on decision making and human reasoning has 
been the focus of a lot of research efforts. The impact of context on the way an individ- 
ual selects an alternative during decision process is well accepted. The same observa- 
tion applies to requirements elicitation and the selection of default requirements. The 
problem of default requirements is based on conflicts between W a and W Engineers' it 
is therefore interesting for engineers to understand the actual content of W Engineers, 
and more specifically of W 'context Elicitation- Since this knowledge is about the con- 
text, there is a need to define what are the important elements inside the context that 
must be discussed with stakeholders. Understanding what elements of the elicitation 
context influence her decision to communicate or not a requirement is therefore an 
interesting question in the scope of RE. 

To achieve such understanding, there is a need to accurately define elements of the 
context that actually influence stakeholder during elicitation. Those elements of con- 
text should be categorized into families, or dimensions of the context, to obtain a solid 
taxonomy of context for RE. The purpose of this section is to review definitions of 
context from ubiquitous and context-aware computing literature: these definitions are 
indeed centered on the notion of information systems, which make them particularly 
relevant considering the aim of the paper. Since lexical definitions or illustrations of 
context are not accurate enough for our purpose of supporting engineers during elici- 
tation, we focus on operational definitions of context. Table [T] summarize the different 
categories of context that are identified in the literature on context (X refers to cate- 
gories that are explicitly treated in the article, ? refers to categories that are suggested, 
but not explicitly mentioned). 



Table 1 : Categories of Context 
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We observe agreement on time and space categories as members of the operational 
definition of context. This agreement generally comes from technological opportunities 
to sense automatically these families (timer, GPS, ...)• The same agreement is observed 
for individuals. Nearby resources are another aspect that is often proposed: it includes 
animals but also nonliving things such as materials, objects, artifacts that can be ac- 
cessed by individuals and with which they may interact. 

While the large majority of definitions deal with "things" and their location in space 
and time, it is much more difficult to find an agreement on the remaining categories that 
form context. The physical conditions are sometimes considered as relevant for the def- 
inition of context: temperature, noise level, light are examples of elements of context 
that can be considered by some context-aware systems. Some operational definitions 
never account for such elements, probably because of their thiner granularity. Knowl- 
edge is an aspect that is only proposed by Lenat: this category includes elements about 
the content of knowledge, its justification and how it is evaluated by individuals. The 
notion of relationship between individuals is another recurrent element in context liter- 
ature: it deals with the relations that exist between two or more individuals. Activity is 
a dimension proposed by Zimmermann et al. which deals with the goals of individuals: 
what do they aim to do in the context? Computer's state and Imaginary companions 
are other plausible elements of context proposed by Brown. Imaginary companions 
are "spirits" to which an individual can attach some notes which represent knowledge 
about this spirit. It offers an interesting hint to represent pieces of information about 
individuals that are not directly related to the context. 

Grouped together, former categories would offer a first raw operational definition 
of context, but would amount to introduce several weaknesses in the framework: re- 
dundant elements, categories with different granularity levels, and a rather unintuitive 
taxonomy for elements of context considered in the scope of elicitation. In the next 
part of the paper, we discuss former categories of context that are important to the re- 
quirements engineers, and propose a context framework, based on our literature review, 
which do not suffer form previously described threats. 

5 Categories of Context 

Based on our literature review, we propose a framework to support engineers during 
elicitation. The framework lists categories of context that are likely to influence stake- 
holders' default selection. It is possible for engineers to use the framework as a check- 
list against which to compare a preliminary requirements documentation, thereby em- 
phasizing aspects of context that have not been considered. Identifying those gaps en- 
ables to identify new questions to ask and make the elicitation more complete, i.e., 
accounting for categories of context eases to reach completeness of elicitation. 

The context framework is a taxonomy inspired by our literature review and adapted 
to include dimensions that influence default selection during elicitation. Those cate- 
gories are (i) Localization; (ii) Items; (iii) Granularity; (iv) Activity; (v) Relationship 
and (vi) Knowledge. Based on these six dimensions, it is possible to account for each 
category proposed in literature on ubiquitous and context-aware computing. These six 
categories have never been proposed together as a single operational definition of con- 
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text. The framework is represented in Figure [3] 
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Figure 3: Context Framework 



To illustrate the way families of the context may impact the default requirements 
selection, we use our introductory example: a stakeholder wants the future software 
to have the feature of sharing data, which presupposes to consider some prerequisites 
such as the need for the program to supports TCP/IP and to be open source. The aspect 
we are interested in is the selection of particular default requirement by stakeholders, 
depending on definition of the elicitation context. We provide some examples - under 
the form of questions - of how changes in one of the six categories of context could 
influence the selection of default requirements. §6 proposes an empirical validation of 
the influence illustrated in this section. 



5.1 Items 

This category forms the ground of the context, its content. Items are all the elements 
accessible inside the context and any information that would disappear if the item is 
destructed. For this category, we adopt Zimmermann et al.'s position suggesting that an 
entity can be of different nature: natural, human or artificial iPPHl . 

(Ex. 1) Will the future system share data with customers' or internal's appli- 
cations? 

The selection of default requirements strongly depends on who - or what - we 
talk about, e.g., sharing data with customers obviously raises the question of security, 
in order to ensure the software shares only adequate data with adequate customers, 
which is not a real concern for intern applications. A single switch of Items creates a 
completely different but still relevant context. These two contexts may potentially lead 
to very different requirements documentation. Item category answers to the following 
question: what are we actually talking about in the given context? 

5.2 Granularity 

This category deals with the nature, the quantity and the level of information that is 
provided about parts of the context. The granularity category applies at two different 
levels. It is related to items and therefore indirectly influences the definition of the 
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context (see Figure[3]l. It gathers elements that describe individuals, e.g., age, address, 
skills, hobbies for a human. It may also describe machines, e.g., state, memory, CPU, 
GUI , etc. or natural entities. It may also apply directly to the context, e.g., environ- 
mental conditions such as temperature, light, noise, and so on. 

(Ex. 2) Will the future system share accurate financial data or raw operational 
data? 

Granularity category refines the situation and modifies the context by providing ad- 
ditional information on a given item (here data) or on the context in which this item 
exists, thereby modifying the selection of default requirements, e.g., working with ac- 
curate financial data should imply transactional processing and quality control require- 
ments which may not be explicitetly communicated to engineers. Granularity category 
answers to the following question: how much detail does the stakeholder know about 
the items or the context itself? 

5.3 Activity 

This category deals with the set of goals and intentions of items existing in the context. 
It can be related to human entities (i.e., winning the championship) or to natural or 
artificial entities (firms which aim to generate profit). It is related to items and therefore 
indirectly influences the definition of the context. 

(Ex. 3) Will the future system forecast production level or simply monitor it? 

Depending on the goals of the future system's users, default requirements are going 
to be different, e.g., frequency of measurement should be higher if the system aims to 
monitor production. Activity corresponds to goals of stakeholders: it is the category 
of context that is typically considered in classical elicitation literature. Nevertheless, 
engineers should bear in mind that activity is not only related to the stakeholder herself, 
but also to other items existing inside the context. Activity category answers to the 
following question: what are the goals of items inside the context? 

5.4 Relationship 

This category deals with the relationships that exist between two or more items, i.e., 
the way some entities are related to each other. It is the last category which is directly 
related to items and which therefore indirectly influences the definition of the context. 

(Ex. 4) Will users of the future system be colleagues with similar levels of re- 
sponsibility or a group of managers and employees, with heterogeneous levels 
of responsibility ? 

The relations that exist between items set constraints that are likely to impact the 
selection of default requirements, e.g., hierarchical differences between managers and 
employees raises questions regarding data access rights. Considering the relationships 
between entities of the context is consequently another aspect that is likely to influence 
the stakeholder in the selection of default requirements. Relationship category answers 
to the following question: how do items from the context relate to each others? 
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5.5 Localization 



This category acts as flags to delimit the position of a context on a lineage or a map. 
More specifically, the category gathers factors related to the time when the context 
occurs or to the place where it is located. They are directly related to the context. 

(Ex. 5) Will the future system be implemented in three month or in three 
decades? In offices in Belgium or in a weather station in Antarctica? 

It is well accepted that what is true today may not be true tomorrow. The same 
assertion applies for location, e.g., telecommunication facilities in Antarctica are likely 
to be different than those in Belgium. There is consequently a need to account for when 
and where the context of the system is localized. Localization category answers to the 
following question: when and where occurs the context?" 

5.6 Knowledge 

This category deals with the knowledge that exists in the context and which is not di- 
rectly related to items, e.g., laws, culture, habits, etc. The category is about the content 
of this knowledge, but also about its justification and its status inside the context. 

(Ex. 6) Why the future system will share data in compliance with a particular 
privacy regulation? Why does it respect some traditions or habits? 

There may exist some content which is not strictly speaking an item - it does not 
have properly goals or relationships - that set constraints on the default selection, e.g., 
the law obviously implies for the stakeholder to consider default requirements that 
would otherwise have been of no great importance. In addition to items, there are con- 
sequently some non-embodied parts of the context that still play a role in the selection 
of default requirements. Knowledge category answers to the following question: what 
pieces of information frame the context? 

6 Experiment 
6.1 Design 

This section describes the empirical study performed to validate our context framework 
as a collection of six categories that are likely to influence how stakeholders select 
default requirements during elicitation. To validate the framework, each family is tested 
through its own questionnaire, which tests several modalities of the category and avoids 
interactions between dimensions. 

Each questionnaire consists of three distinct sections: a context statement, the as- 
signment and the variability area. Firstly, subjects are asked to carefully read the con- 
text statement. Secondly, an assignment explains the problem to be solved and the dif- 
ferent solutions that are available. Thirdly, based on the referent context, subjects have 
to provide answers to the problem, according to the referent context, but also according 
to different modalities of a family introduced in the variability area. 
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Subjects are explicitly told that they can adapt their answers depending on the 
modality, and that change is not mandatory, e.g., the "Relationship" questionnaire in- 
troduces a referent context: it explains that there is a regular competition between items 
A and B. Subjects first have to solve a problem considering this referent context. Then, 
the variability area asks subjects if they would adapt their answers, considering that a 
"very strong competition" or a "collaborative behavior" is observed between the two 
items. Subjects can confirm their initial choice, and behave toward variation as they 
behaved toward referent context, or they can adapt their choices, thereby suggesting 
influence of context on default based decision making. 

6.2 Structure of Questionnaires 
6.2.1 Context Statement 

Th context statement describes a basic context to subjects. The context consists of a 
regular benchmark problem requiring default reasoning, and will be referred to as ref- 
erent context. A benchmark is a problem that systematically introduces at least two 
objects that are supposed to respect a rule, then informs the subject one object does 
not respect the rule, i.e., there is an exception to the rule, and finally asks the sub- 
ject whether the remaining object respects that rule. It has been designed based on 
a managerial decision making issue. Subjects are introduced to details about Items, 
Relationships, Activity, Granularity, Localization and Knowledge elements. These el- 
ements remain unchanged throughout the entire experiment, e.g., subjects answering 
the "Localization" questionnaire face the same referent context as subjects answering 
the "Knowledge" questionnaire. 

We ensure each element of the referent context is easily adjustable. The only aspect 
that changes between the six questionnaires is the definition of the category we want 
to test. It is necessary that each referent element can be replaced by substitute elements 
presented in the variability area. The referent context used in our experiment is: 

"A survey established by a Belgian national daily newspaper, is pub- 
lished on January 31, 2009. The survey proposes a comparison of some 
logistic means. Among others, it compares the supply of goods by rail or 
by boat. The newspaper asserts both are means of transport with transport 
areas of their own. The newspaper also asserts that types of transport that 
have their own infrastructures (waterways, railways, etc.) usually offer a 
reliable service: little delay, no risk of damage, etc. Finally, the newspa- 
per emphasizes that these types of transport were originally created for 
logistic, which has led to an incredibly strong competition between them. 

A year later, in 2010, the newspaper puts forward that for some time, 
deliveries by train arrived regularly with a delay and more significant 
losses. The newspaper has not had the opportunity to repeat a study about 
the boats. As a new manager in a logistics company, it is your duty to 
decide whether you can trust the boat transport." 

The referent context can be summarized with the following default benchmark 
problem: logistic means with transport areas of their own provide typically good ser- 
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vice quality (default rule). Trains (object l) and boat (object 2) have transport areas of their 
own. Trains provide poor service quality (exception). What quality is provided by boats? 

(benchmark) 

6.2.2 Answers 

Once the referent context is presented, the questionnaire asks subjects to make a deci- 
sion. Since the referent context is built according to the benchmark problem structure, 
the choice offered to subjects is limited to four different options. For each element 
inside the variability area, subjects are asked to select one (and only one) answer. 

• Benchmark: the exception has no bearing on the remaining object, e.g., object 2 
respects the default rule; 

• Exception: the exception also applies to the remaining object, e.g., object 2 is 
also an exception to the default rule; 

• Other: the exception does imply another exception, but with different charac- 
teristics, e.g., object 2 does not respect default rule because it does not exist 
anymore; 

• Can't Say: the subject cannot choose one of the former propositions. 

6.2.3 Variability Area 

The variability elements are introduced in the last part of the questionnaire. The last 
section asks subjects to answer to the question "what about the trains?", considering 
turn by turn each of the n modalities of the category we want to test: 

• Element 1: the referent modality of the family. The modality of the category that 
is presented in the referent context; 

• Element i(i = l...n— 1): a substitute modality that modifies the definition of the 
referent context. 

The n parameter has been carefully considered since a trade-off exists: little n re- 
duces the possibility to interpret the influence of category changes; large n makes the 
questionnaires too long, with a risk a bias/lack of motivation of subjects. In our exper- 
iment, n varies between 4 and 6. It is worthy to note that a same questionnaire always 
alters at most one category. Thereby, if an impact on default selection is observed, it 
can only be explained by the varying category. 

6.3 Subjects 

A total of 260 subjects were questioned. All subjects were students in management 
sciences, economics or computer sciences from the University of Namur. On average, 
each subjects answers five different questions: this brings the number of treated data to 
approximatively 1300 decisions. Details about respective size of samples are presented 
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in the results section. Each group had twenty minutes to read the referent context def- 
inition and answer the questions. Subjects were not compensated for participating in 
the study, and were asked to answer during class time. 

6.4 Procedure 

Subjects can use any kind of material. The assignment clearly mentions there is no 
best answer, but that some answers are better than others. It also tells subjects that the 
objective of the questionnaire is to understand how managers reason in a situation of 
general and imperfect information about the decision problem they are facing. 



7 Results 

Significance tests presented in this section are performed using a repeated measure 
ANOVA on the proportion of answers, in each category of answer and for each prob- 
lem. Such approach implies that the category of answer is considered as another factor. 

The validation of our context frameworks passes by the observation that, given a 
change in the decision making context, there is a change in the outcome of the decision 
process. Consequently, observing significant differences in proportion of answers, for 
each category and for each modality, would mean our framework is valid to determine 
important elements of context to consider during elicitation. 



7.1 Items 

We modify the referent context of this questionnaire to avoid any reference to objects' 
name. Instead of names, we prefer lexical items with no particular meaning in order 
to avoid direct pointers to items. A typical formulation of the referent context is: "it 
compares the supply of goods by X or by Y". The variability area proposes different 
values - or modalities - for X and Y: 



1. X is a train and Y is a boat; 

2. X is a new teleportation technology and Y is a plane; 

3. X is a helicopter and Y is a plane; 

4. X is a sub-contractor and Y is the national postal service. 

We collected 54 questionnaires for this category. Proportions of answer are repre- 



sented in Figure 4a We observe significant differences between the proportions of an- 
swer for each modality. There is a significant effect of the answer category [P(F(3, 159)> 
8.29 )= 0.000] and a significant two-way interaction between answer category and 
modality [P(F(9,477)> 1.958 )= 0.0424]. The referent context is considered by sub- 
jects as the regular situation, and subjects do better for this modality than for the other 
ones (53.7% of Benchmark). We observe a different pattern of answer for modalities 
involving artificial items (mod. 2) and for real items (mod. 3 & 4). 
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1 


2 


3 


4 


Benchmark 


.537 


.370 


.278 


.352 


Abstention 


.074 


.204 


.148 


.130 


Exception 


.167 


.111 


.296 


.278 


Other 


.222 


.315 


.278 


.241 



Table 2: Individualities - Proportion of Answers 



7.2 Granularity 



We modify the referent context so that details about items never appear. The variability 
area proposes different modalities for details that are provided regarding the exception 
and the items. The sentence "the article puts forward that for some time, deliveries by 
train arrived regularly with a delay X" will be the center of our attention for this second 
questionnaire. The variability area proposes the following modalities for X: 

1. ...of [on average] one our and very important damages; 

2. ...of [on average] ten minutes and moderate damages; 

3. ...of [on average] one day, untrained container handlers, and unacceptable losses; 

4. The articles also emphasize similarities between the two means of transport; 

5. The articles also emphasize differences between the two means of transport. 

We collected 32 questionnaires for this category. Proportions of answer are repre- 



sented in Figure 4b We observe significant differences between the proportions of an- 
swer for each modality. There is a significant effect of the answer category [P(F(3,93)> 
18.31 )= 0.000] and a significant two-way interaction between answer category and 
modality [P(F(12,372)> 11.73 )= 0.000]. The referent context is considered by sub- 
jects as the real world situation, and subjects do better for this modality than for the 
other ones (78.1% of Benchmark). We observe two different patterns of answer: the 
first one for specificity factors (mod. 1,2 & 3) and the second one for similarity factors 
(mod. 4 & 5). 





1 


2 


3 


4 


5 


Benchmark 


.781 


.406 


.656 


.063 


.406 


Abstention 


.094 


.125 


.063 


.094 


.125 


Exception 


.063 


.312 


.125 


.781 


.031 


Other 


.094 


.156 


.156 


.062 


.438 



Table 3: Granularity - Proportion of Answers 



7.3 Activity 

We modify the referent context so that details about the purpose - or the activities - of 
items existing inside the context never appear. In this questionnaire, the sentence "Fi- 
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nally, the newspaper emphasize that these types of transport were originally created/or 
logistic" that normally appears in other questionnaires is removed. We replace the sen- 
tence by elements inside the variability area and then ask subjects to draw a conclusion. 
The variability area proposes the following modalities: 

1 . these types of transport were originally created for freight; 

2. these types of transport were originally created to transport persons; 

3. these types of transport were originally created to transport any kind of load; 

4. these types of transport were originally created for leisure. 

We collected 45 questionnaires for this category. Proportions of answer are repre- 



sented in Figure 4c Subjects did not agree on the referent context as the real world 
version of the context, and they do not reach the best performance for this modality 
(only 35.6% of'Benchmark"). Subjects prefer mod. 3 as the real world version, and 
reach 40% for mod. 2 & 3. We observe a significant effect of the answer category 
[P(F(3,132)> 6.319 )= 0.000], and a significant two-way interaction between answer 
category and the modality of activity family [P(F(9,396)> 2.756 )= 0.004]. 





1 


2 


3 


4 


Benchmark 


.356 


.400 


.400 


.133 


Abstention 


.356 


.333 


.333 


.422 


Exception 


.111 


.111 


.156 


.067 


Other 


.178 


.111 


.111 


.378 



Table 4: Activity - Proportion of Answers 



7.4 Relationship 

We modify the referent context so that details about the relationships that exist be- 
tween items inside the context never appear. In this questionnaire, the second part of 
the sentence "Finally, the newspaper emphasize that these types of transport were orig- 
inally created for logistic, which has led to an incredibly strong competition between 
the two types of transport'" that normally appears in other questionnaires is modified. 
We replace it by elements inside the variability area and then ask subjects to draw a 
conclusion. The variability area proposes the following modalities: 

1 . which has led to an incredible competition between the two transport; 

2. which has led to an regular competition between the two transport; 

3. no information are available about the relationship between the two transport; 

4. which has led to a strong collaboration between the two transport; 

5. which has led to a weak collaboration between the two transport; 

6. which has led to a neutral position between the two transport. 

We collected 32 questionnaires for this category. Proportions of answer are repre- 
sented in Figure |4d] Subjects agreed on the referent context as the real world version 
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of the context. They behave accordingly by reaching a 50% rate of "Benchmark" an- 
swers. We observe a significant effect of the answer category [P(F(3,93)> 3.991 )= 
0.010] and a significant two way interaction between answer category and the modal- 
ity of relationship family [P(F(15,465)> 3.967 )= 0.000]. 





1 


2 


3 


4 


5 


6 


Benchmark 


.500 


.375 


.187 


.125 


.219 


.313 


Abstention 


.250 


.375 


.656 


.313 


.219 


.500 


Exception 


.094 


.094 


.094 


.406 


.188 


.031 


Other 


.156 


.156 


.156 


.156 


.375 


.156 



Table 5: Relationship - Proportion of Answers 



7.5 Localization 

We modify the referent context so that details about the place where - and the time when 
- the context occurs never appear. The referent context is supposed to occur in Belgium, 
nowadays. We replace these information by elements inside the variability area and 
then ask subjects to draw a conclusion. The variability area proposes the following 
modalities: 

1. The context occurs in Belgium, nowadays; 

2. The context occurs in China, nowadays; 

3. The context occurs next to my home, nowadays; 

4. The context occurs in 1948; 

5. The context occurs in 2025; 

6. The context occurs in China, in 2025. 

We collected 45 questionnaires for this category. Proportions of answer are repre- 
sented in Figure |4e] Subjects mostly agreed on the referent context as the real world 
version of the context, and behave accordingly by reaching a 43.5% rate of "Bench- 
mark" answers. We observe a significant effect of the answer category [P(F(3,93)> 
3.991 )= 0.010] and a significant two way interaction between answer category and the 
modality of time and space family [P(F(15,465)> 3.967 )= 0.000]. 





1 


2 


3 


4 


5 


6 


Benchmark 


.435 


.326 


.283 


.239 


.065 


.130 


Abstention 


.109 


.239 


.239 


.261 


.348 


.457 


Exception 


.152 


.087 


.130 


.239 


.130 


.087 


Other 


.304 


.348 


.348 


.261 


.457 


.326 



Table 6: Time and Space - Proportion of Answers 
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7.6 Knowledge 



We modify the referent context so that details about the justification of the knowledge 
that exists inside the context never appear. The referent context normally states that 
knowledge is issued from "A survey established by a national daily newspaper". We 
replace this information by elements inside the variability area and then ask subjects to 
draw a conclusion. The variability area proposes the following modalities: 

1. The survey is established by a national daily newspaper; 

2. The survey is established by a national management journal; 

3. The survey is established by an international management journal; 

4. The survey is established by a local people magazine. 

We collected 52 questionnaires for this category. Proportions of answer are repre- 
sented in Figure|4f| Subjects did not agreed on the referent context as the real world ver- 
sion of the context. They preferred the national management journal as the real world 
version, and behave accordingly by reaching a maximum of 38.5% rate of "Bench- 
mark" answers for modality 2. We observe a significant effect of the answer category 
[P(F(3,153)> 5.097 )= 0.002] and a significant two way interaction between answer 
category and the modality of knowledge family [P(F(9,459)> 3.493 )= 0.000341]. 





1 


2 


3 


4 


Benchmark 


.327 


.385 


.269 


.173 


Abstention 


.308 


.250 


.288 


.500 


Exception 


.135 


.115 


.019 


.154 


Other 


.231 


.250 


.423 


.173 



Table 7: Knowledge - Proportion of Answers 



8 Discussion 

The objective of our experiment is not to study the direction of variations of decision 
behaviors when the definition of an initial context is changing: we do not aim to explain 
why subjects choose the "Benchmark" answer in modality 1 and prefer the "Exception" 
answer in modality 2. Such work, though interesting, would require many empirical 
efforts over the long term. The objective of our experiment should rather be expressed 
at a higher level. We aim to determine whether subjects change their choice behavior 
for a same context, when one family of the context framework is changing over time. 

The experiment we describe in this paper enables to demonstrate the real impact of 
the six categories of context on decision making, thereby validating the context frame- 
work as an adequate tool to consider context during the elicitation of requirements. 
This conclusion is achieved based on two major observation. Firstly, we observe that 
subjects systematically reach the maximum "Benchmark" proportion for the context 
that they consider to be the most representative of the real world. We interpret this 
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Figure 4: Results of the Experiment by Categories 
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as consistency, i.e., subjects are less likely to be influenced by elements of the con- 
text when these elements fit their own perception of the real world. More formally, the 
more elements defining context fit default assumptions, the more subjects choose the 
"Benchmark" answer. Secondly, we observe that subjects systematically decrease their 
performance when the context is changing according to one dimension. We do not ex- 
plain why their modify their choice, we simply observe that their significantly do so. 
We do not test the interaction between categories, we test the influence of standalone 
varying categories. This shows the importance of context for decision makers. 

9 Conclusion 

The paper studies decision making during elicitation of requirements, and how to 
achieve better completeness of elicitation. Based on default logic, we propose a dis- 
tinction between ground requirements that are explicitly communicated and default re- 
quirements that are implicitly assumed but not always correctly communicated to - or 
identified by - engineers. We argue the selection of default requirements is influenced 
by the context's definition of the system. We consequently review definitions of con- 
text proposed in context-aware literature. The paper then proposes a context framework 
that supports engineers in the identification of default requirements. We argue that two 
slightly different contexts are likely to lead to very different default requirements selec- 
tions. The context framework consequently forms a checklist against which to perform 
the elicitation, to make sure each aspect of the context has been considered. The paper 
proposes an empirical validation of the framework. We observe that each of the six 
categories of our framework significantly influences the way individuals make default 
based decisions. The framework therefore constitutes an adequate tool to support en- 
gineers in the identification of questions to ask to stakeholders during elicitation. The 
framework is also a well structured definition of context that fits the needs of elicitation 
and opens the way to further understanding of decision making during elicitation. 
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